Exploring data¶

When working with a dataset, the first step is usually to understand what data and metadata it contains. In this chapter we explore how scipp supports this.

This tutorial contains exercises, but solutions are included directly. We encourage you to download this notebook and run through it step by step before looking at the solutions.

First, in addition to importing scipp, we import scippneutron since this is required for loading Nexus files:

[1]:

import scipp as sc
import scippneutron as scn
import numpy as np

We start by loading some data (download here), in this case measured with a prototype of the LoKI detectors at the LARMOR beamline:

[2]:

data = scn.load(filename='LARMOR00049338')

Workspace run log 'good_frames' has unrecognised units: 'frames'
Workspace run log 'raw_frames' has unrecognised units: 'frames'

Note that the exercises in the following are fictional and do not represent the actual SANS data reduction workflow.

Step 2: Plot the data¶

Scipp objects can be created using the plot() method. Alternatively sc.plot(obj) can be used. Since this is neutron-scattering data, we can also use the “instrument view”, provided by scn.instrument_view(obj) (assuming scippneutron was imported as scn).

Plot the loaded data and familiarize yourself with the controls.
Create the instrument view and familiarize yourself with the controls.

[4]:

data.plot()

[5]:

scn.instrument_view(data)

Step 5: A closer look at the data¶

The 2-D plot we obtain above by default is often not very enlightening. Define:

[15]:

counts = sc.sum(data, 'tof')

Exercises: 1. Create a plot of counts and also try the instrument view. 2. How many counts are there in total, in all spectra combined? 3. Plot a single spectrum of data as a 1-D plot using the slicing syntax to access the spectrum.

[16]:

# slice is optional, making plot more readable in the documentation
counts['spectrum', 56000:62000].plot()

[17]:

scn.instrument_view(counts, norm='log')

[18]:

# sc.sum(counts, 'spectrum') # would be another solution
sc.sum(data).value

[18]:

58142417.0

[19]:

data['spectrum',10000].plot()

As seen in the instrument view the detectors consist of 4 layers of tubes, each containing 7 straws. Let us try to split up our data, so we can compare layers. There are other (and probably better) ways to do this, but here we try to define an integer variable containing a layer index:

[20]:

z = sc.geometry.z(data.coords['position'])
near = sc.min(z)
far = sc.max(z)
layer = ((z-near)*400).astype(sc.dtype.int32)
layer.unit = ''
layer.plot()

Exercises: - Change the magic parameter 400 in the cell above until pixels fall cleanly into layers, either 4 layers of tubes or 12 layers of straws. - Store layer as a new coord in data. - Use sc.groupby(data, group='layer').sum('spectrum') to group spectra into layers. - Inspect and understand the HTML view of the result. - Plot the result. There are two options: - Use plot with projection='1d' - Use sc.plot after collapsing dimensions, sc.collapse(grouped, keep='tof') - Bonus: When grouping by straw layers, there is a different number of straws in the center layer of each tube (3 instead of 2) due to the flower-pattern arrangement of straws. Define a helper data array with data set to 1 for each spectrum, group by layers and sum over spectrum as above, and use this result to normalize the layer-grouped data from above to spectrum count.

[21]:

# NOTE:
# - set magic factor to, e.g., 150 to group by straw layer
# - set magic factor to, e.g., 40 to group by tube layer
layer = ((z-near)*150).astype(sc.dtype.int32)
layer.unit = ''
data.coords['layer'] = layer
grouped = sc.groupby(data, group='layer').sum('spectrum')
grouped.plot(projection='1d')
sc.plot(sc.collapse(grouped, keep='tof'))

[22]:

norm = sc.DataArray(data=layer*0+1, coords={'layer':layer})
norm = sc.groupby(norm, group='layer').sum('spectrum')
sc.plot(sc.collapse(grouped/norm, keep='tof'))

Exploring data¶

Step 1: Use the HTML representation to see what the loaded data contains¶

Step 2: Plot the data¶

Step 3: Exploring meta data¶

Step 4: Fixing metadata¶

Step 5: A closer look at the data¶